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PERFORMANCE MONITORING SYSTEM AND METHOD 

The invention relates to a performance monitoring system and method. In 
particular, although not exclusively, the invention relates to a system and method 
5 for monitoring the performance of equipment operators, particularly operators of 
draglines and shovels employed in mining and excavation applications or the like. 

BACKGROUND TO THE INVENTION 
1n many fields of manufacturing and Industry, it is desirable or necessary to 

10 monitor the performance of equipment operators in addition to the equipment itself. 
This may be for managerial purposes to ensure that operators are complying with a 
minimum required standard of performance and to help Identify where 
improvements in performance may be achieved. Monitoring performance may also 
be desired by an operator to provide the operator with an indication of their own 

15 performance in comparison with other operators and to demonstrate their level of 
competence to management 

One field In which performance monitoring is required Is the operation of 
drag fines and shovels and the fflce as used in large-scale mining and excavation 
applications. For commercial purposes, it is important that an operator is operating 

20 a piece of machinery to the best of the operator's and the machine's capabilities. 

There are however many factors that need to be measured and considered 
to enable fair and useful comparisons to be made between different operators, 
between different machines, between present and previous performances and 
between different operating conditions. 

25 It Is therefore desirable to provide a system and/or method capable of 

achieving this objective. Furthermore, it Is desirable that performanoe-frronftoring 
information Is promptly available to inform management and operators alike of 
current performance. 

30 DISCLOSURE OF THE INVENTION 

According to one aspect, although it need not be the only or indeed the 
broadest aspect, the invention resides in a method for monitoring performance of 
at least one machine operator, the method including the steps of ; 
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measuring at least one machine parameter during operation of the machine 
by the operator; 

generating at least one performance indicator distribution from 
measurements of the at least one machine parameter; and. 

calculating at feast one performance indicator from the at least one 
performance Indicator distribution. 

The method may further include the step of providing feedback to the 
operator by displaying the at least one performance indicator in substantially real- 
time to the operator. Alternatively, the at least one performance indicator may be 
displayed to the operator once the machine has completed an operation cycle. 

Suitably, the at least one machine parameter may be a dependent machine 
parameter. Alternatively, the at least one machine parameter may be the sole 
parameter represented by a particular performance Indicator. 

The method may further include the step of segmenting at feast one of the 
dependent maohine parameters into segments, the range of each segment 
constituting a segmentation resolution. 

Suitably, the step of segmenting at feast one of the dependent machine 
parameters includes specifying a magnitude of the range for each segment of each 
dependent machine parameter requiring segmentation. 

Suitably, at feast one dependent machine parameter may not require 
segmentation. 

Suitably, the step of generating the at feast one performance indicator 
distribution may comprise using a mixture of one or more distributions to model the 
performance indicator distribution. The number of mixtures may be set dynamlcaJy. 

Suitably, the at feast one performance indicator distribution may be 
generated using an algorithm. The algorithm may be an LBG algorithm. 
Alternatively, the at feast one performance indicator distribution may be generated 
using a linear ranking model (LRM). 

Suitably, two or more performance indicators may be combined to yield an 
overall performance rating of the machine operator. One or more of the 
performance Indicators may be positively or negatively weighted with respect to the 
other performance Indicators). 

According to another aspect, the Invention resides in a system for 




WO 03/063032 PCT/AU03A)OO77 

3 

monitoring performance of a machine operator, the system comprising; 

at least one measuring device for measuring at least one machine parameter 
during operation of the machine byttie operator; 

a server for generating at least one performance Indicator distribution from 
5 measurements of the at least one machine parameter, and, 

a performance indicator calculation module for calculating at least one 
performance Indicator from the at least one performance Indicator distribution. 

Preferably, the server is remote from the machine. 

Suitably, the server comprises storage means, communication means and 
10 a performance indicator distribution calculation module. 

Suitably, the performance indicator calculation module is onboard the 
machine. 

Preferably, the performance Indicator calculation module Is coupled to 
communication means for transmitting and receiving data to and from the server. 

15 Preferably, the system further comprises at least one display device for 

displaying the at least one performance indicator in substantially real-time to the 
operator. Alternatively, the at least one performance Indicator may be displayed to 
the operator once the machine has completed an operation cycle. The at least one 
display device may be situated in, on or about the machine and/or remote from 

20 the machine. 

Suitably, the communication means comprises a transmitter and a 
receiver. 

Further aspects of the Invention will become apparent from the following 
description. 

25 

BRIEF HP^RIPTION OF THE DRAWINGS 
To assist in understanding the invention and to enable a person skilled in 
the relevant art to put the Invention Into practical effect preferred embodiments 
will be described by way of axample only end with reference to the accompanying 
30 drawings, wherein: 

FJG 1 shows a distribution of data representing a production Key 

performance indicator ( KPI); 

FIG 2 is a schematic plan view of a machine showing segmentation 
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resolution for the swing angle parameter; 

FIG 3 shows a distribution of Fill Production KPI data; 

FIG 4 shows dragline data for the parameters start fill reach versus start fill 

height; 

5 FIG 5 shows calculation of a KPI for the right side of the distribution; 

FIG 6 Is a schematic representation of an Integrated Mining Systems 
(IMS) system structure employed In the present invention; 

FIG 7 shows a display of KPIs showing current real-time performance and 
a comparison with performance for a previous cycle; 
10 FIG 8 shows a display of KPIs showing current real-time performance; 

FIG 9 shows an alternative display of KPIs showing both current real-time 
performance and performance for a previous cycle; 

FIG 10 shows an Operator Performance Trend Report; and 

FIG 1 1 shows an Operator Ranking Report 

15 

DETAILED DESCRIPTION OF THE INVENTION 
The present Invention montors one or more parameters or variables of a 
machine to provide an accurate indication of how well an operator is performing, for 
example, in comparison with other operators for the same machine and/or m 

20 comparison wtth previous performances of the same operator. 

Although the present invention will be described in the context of monitoring 
the performance of machines found on a mining site, ft wiD be appreciated that the 
present invention is applicable to a wide variety of machines found in various 
situations and performance monitoring is required. 

25 A machine parameter may itself be referred to as a key performance 

indicator (KPI)- Alternatively, a KPI may be dependent on one or more machine 
parameters. The KPIs may be represented and displayed as a percentage or a 
score, such as points scored out of 10, that describes how well the operator is 
performing for a given parameter and/or KPI. A high percentage value, such as 

30 >90% for example, shows that the operator is performing extremely well. A mid- 
range value for a KPI, such as 50% for example, shows that the operator's 
performance is about average and less than this example percentage 
demonstrates that their performance is below average for that KPI. 
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Each KPI parameter is related to the performance of an operator for one or 
more given machine parameters such as fin time, cycle time, dig rate, and/or 
other parameters). KPIs are a measure of how the operator is performing for the 
particular parameters) related to that KPI compared to the other operators. The 
5 performance of, or rating for, a particular operator is calculated using, in part 
previous data recorded for the machine and provides an indication of whether or 
not the operator is improving. The process for measuring the parameter* and 
achieving the KPIs is described in detail hereinafter. 

The parameter data is acquired using conventional measuring equipment 
10 such as sen sore, timing means and the like and the particular equipment required 
to acquire the data would be familiar to a person of ordinary skill In the relevant 
art 

Different comparisons between the data are also posatole. The current 
operator of a machine can be compared to an the other operators of the same 
15 machine or to the operators previous perfbrmance(e). This shows how well they 
perform against others and shows them whether they are improving respectively. 

One important consideration of the present invention is filtering the data 
from ail the machines that may be present in, for example, a mine site or other 
situation to enable fair and meaningful comparisons to be made* Various factors 
20 that may affect KPI parameters are as follows: 

Machine: Each machine possesses different operating characteristics 
and therefore the data from one machine wil not reflect the performance of 
operating another machine. 

Dig Mode: Different dig modes are possible with a single machine and 
25 these may differ between different machines, which is significant In the present 
invention operators can enter a particular dig mode corresponding to the mode of 
operation of the machine. The selected dig mode must be correct otherwise the 
KPIs may be mis-represented and provide misleading results. 

Operator Operators can compare their performance against their own 
30 previous performances to verify whether they are improving. Operators can also 
compare their performances against those of other operators. 

Location: Different locations in the mine wiH have different digging 
conditions even though the digging mode may be the same. This may be 
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represented by the specific gravity (s.g.) or by an index that describes the current 

digging difficulty, known as the dig index. 

Bucket Some KPls will be affected by the type of bucket being used 

on the dragline. For example, different size buckets, which are usually pre- 
5 selected on the basis of the application, may produce different dig rates. For 

comparison purposes, an operator should not be disadvantaged when using a 

smaller bucket ■ 

Bucket Rigging: If this factor changes, but the bucket does not, the KPI 

results may be affected. 
10 Weather. The weather can change the digging conditions and 

therefore affect the performance attained by the operator. 

Some of the above parameters are readily filtered from the data, such as 
machine, dig mode, operator, bucket and possibly location. The more the data is 

15 divided however, the more data needs to be processed, stored and transmitted 
from the server 8 to the onboard computer module 4 (shown in FIG 6). to 
implement the KPls, To reduce this volume of data, the location parameter could 
optionally be omitted, slnoe location data is generally reflected In the bucket type 
being used. Weather and bucket rigging are more difficult to filter. Therefore, the 

20 parameter filters of machine, dig mode and bucket remain. These parameter 
fitters may be combined with the operator parameter filter. 

If the data of all operators are to be compared, the operator filter is 
omitted. When filtering by operator, the number of operators multiplies the 
amount of date for the mine comparison. For example, if there are 1000 bytes of 

25 KPI data to download to the module for the mine data and there are 100 
operators, then this equates to a total of 101,000 bytes of KPI data to download, 
which represents 100 data sets for 100 operators plus one data set for the all 
operator comparison. 

This largo data problem is one of the problems addressed by the present 

30 invention, which enables the present Invention to provide substantially real-time 
monitoring of operators' performance. 

The large data problem can be solved In a number of ways. One option is 
to only download KPI data for the operators that exist fn the recorded data in the 
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database. Alternatively, only KPI data for operators that have over logged onto a 
particular machine, which to stored in an operator profile, may be downloaded. 
For any new operator who logs on, the data b requested and downloaded. If the 
data does not exist in the database, then the display can show that there is no 

5 KPI data for that operator. Another alternative is to just download the KPI data 
for the operator that just logged on. 

Even with the data filtering descrtoed above, a single value such as fill 
time, cannot be compared to other fill times unless one or more dependencies 
are Introduced. Some KPIs, such as the Machine Reliability KPI, do not require a 

10 dependent parameter, but many do, such as the Swing Production KPI, A 
dependent parameter adds another level of filtering to the data that is specific to 
the parameter being rated. 

A simple example is the Swing Production KPI. The time taken to swing a 
dragline, for example, is directly related to the angle through which the dragline 

15 swings (Swing Angle) and the vertical distance the bucket travels from the end of 
a AH to the top of a dump of the bucket contents. These dependencies are 
Included In the KPI calculation by segmenting each of the dependent parameters 
into ranges. The range of the segment is called the segmentation resolution. The 
swing angle in this example could be divided into 10-degree Increments over, for 

20 example, 360 degrees. If the vertical travel distance is ignored in this example, 
this would provide 36 data segments. 

To calculate the KPI, the data recorded from that machine is sorted, for 
example, by dig mode, for each of the segments. For the date associated with 
each segment a KPI distribution Is calculated. Therefore, for the Swing 

25 Production KPI example, the swing times for each swing angle segment are 
extracted and a distribution of times is calculated for each segment Thus, 36 
distributions would be calculated in total. The actual swing times and swing 
angles are measured onboard the machine using conventional timing and angle 
measuring instruments that are familiar to those skilled in We relevant art The 

30 distribution associated with the swing angle segment being measured is then 
selected to calculate the KPI. 

Introducing more dependent variables creates the problem of producing 
more data segments, which in turn means more distributions and more date, In 



WO 03/063032 PCT/AU03/00077 

8 

the example above, if the vertical distance was included and divided into, for 
example, 10 metre segments from 0 to +70 metres (7 segments), there would be 
252 (36 x 7) distributions to calculate and download to the machine Just for the 
Swing Production KPI. 
5 The volume of data oan be reduced by carefully designing the 

segmentation of the dependent parameters, One way Is to Include extremities in 
the segmentation, which allows only segmentation of the arras that are common. 
In the above example, the swing angle could be re-eegmented such that one 
segment contains swing angles less than, far example, 30 degrees and another 

10 segment contains swing angles greater than, tor example, 200 degrees whilst 
maintaining the 10-degree segments between 30 degrees and 200 degrees. This 
re-segmentation results In 19 segments for the swing angle parameter compared 
with 36 in the previous example. 

The vertical height dependency could be reduced to 2 segments by 

15 identifying the height at which the swing velocity is reduced (Le„ for hoist 
dependent swings). Less than this height is one segment and above this height is 
another. This reduces the total number of segments to 38 (2x19) segments. 

As described in the foregoing, a distribution exists for each segment of the 
KPI that Is dependent on some other parameter. Finding a distribution that 

20 describes the KPI data is not trivial. Even though the sampled data looks 
Gaussian in nature, the graphs are skewed and comprise some data at the 
extremities. 

FIG 1 shows some data taken Apr the KPI representing production. All the 
other KPIs show a similar distribution., FIG 1 shows a positive skew in the data 

25 and some data to the right of the graph. A simple Gaussian would model most of 
this data quits adequately. However, it cannot be judged how the data will skew 
or how the distribution wffl change once the KPI Information is available to the 
machine operator, It is likely that the distribution will become more positively 
skewed and less Gaussian like. 

30 One solution to thfe problem Is to model the data with a multi-modal or 

mufti-variant Gaussian mixture in which a mixture of different Gaussian 
distributions are used to model each KPI distribution. This has the advantage that 
the number of mixtures can be changed depending on the data. If the data is very 
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Gaussian-like, then a single mixture comprising a simple Gaussian distribution 
may be used. If the data to very obscure, then a plurality of mixtures can be used 
to describe the distribution* 

The number of mixtures depends on the data that is being modeled and 

5 the number of mixtures may be set dynamically. With sufficient data, an algorithm 
could be employed to determine the maximum number of mixtures required to 
represent the KPI distribution. If there is only a small amount of data, for example 
less than a selectable threshold of 10 samples, then modeling may be carried out 
using a single mixture. If the algorithm does not converge with the maximum 

10 number of mixtures, the highest number of mixtures that cause the algorithm to 
converge can be used. 

One algorithm that could be used to generate the KPI distributions from 
. the data is a Unde-Buzo-Gray (LBG) algorithm, which is known to persons ski Bed 
in the relevant art The LBG algorithm is an iterative algorithm that splits data into 

15 a number of dusters. The algorithm Is designed for vectors, but In the present 
Invention, single dimension vectors (single values) are used, thus simplifying the 
algorithm. 

The detail of the LBG algorithm win now be described, x. ={x l> x 2 
is the training data set consisting of M data samples. C M = {c J9 c 2 ,...,<v} ere the 
20 centrokJs calculated for N clusters, e Is the Iteration conversion coefficient, which 
Is usually fixed to a small value greater than zero, such as 0.01. 

The steps for generating the KPI distributions are as follows: 
1 . N=1 and given X , calculate the initial centrold C x by calculating the mean: 




25 2. Calculate the initial distortion of the data for the initial centrold: 

3. Set iteration Index I «=0. 

4. Find the cluster p with the maximum distortion. 

5. increment the number of dusters: N = N +1 
30 6. Split cluster pinto 2: 
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7. For all 1 £ m £ M In the data set X . record the nearest centrokJ c$ where 
n w is the index of the centrold: 

and the total number of values assigned to each centrold T n . 

8. Calculate the new centroHs: 

10 9, + 

10. Calculate the average of the minimum distortion between the data sample 
and Ha closest centrold: 

11. If {d™-D&)/D%? > s . then go back to step 7. 

IS 12. Save the temporary calculation centrokte In a secure location. 

13. If the number of desired dusters has not been reached, then go back to 
Step 4. 

The algorithm starts by treating the whole of the data as one duster, ft 
then divides the cluster into two and rteratvely assigns data to each of the 

20 dusters until the centrotds of the clusters do not move appreciably. Once the 
Iterations converge, the duster with the greatest spread (accumulative distance 
between data and centrold) Is sptt and the iterative calculations are repeated. 
The algorithm continues until the required number of clusters has been reached. 
The result Is data divided Into dusters with centrokls. The data for each duster is 

25 then used to calculate a mean and standard deviation for that cluster, i.e. a 
distribution. The weight of each duster is calculated as the number of data 
samples in the duster compared to the total number of data samples. This weight 
is known as the mixture ooeffldent 

In order to calculate the KPI from the distributions, the following formula for 
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a mum-variant Gaussian distribution is employed: 

where p(x) is the probability, C„ Is the mixture coefficient and hKx^cx) is 
represented by the following formula: 

i 4^*Y 



5 which Is a standard Gaussian distribution with mean \x and standard deviation a. 

Another solution to the problem of modeling the data to generate the KPI 
distributions is to use a Linear Ranking Model (LRM), Instead of modeling the 
distribution erf each of the segments for each KPI, the LRM models the 
distribution in such a way that only the minimum and maximum boundaries need 

10 to be calculated. Ad values between these Emits are then ranked according to 
their position between the minimum and maximum. This method has the 
advantage that it is distribution independent 

One problem with the LRM is that is does not handle outlying data very 
well. For example, with reference to the FID Production data shown in FIG 3. 

15 there is an amount of data to the right of the graph (caused possWy by abnormal 
cycles). The minimum and maximum values respectively on the abscissa are 
0.33 and 34 (units =mass per unit time interval) for this example. This means that 
the majority of the operators would obtain a low score and very few would obtain 
a high one since the majority of FIB Production values would occur In the lower 

20 half of the range. 

A solution to this problem is to filter off the erroneous data. This can be 
achieved by removing data that is more than 3 standard deviations from the 
mean (keep 99% of the data for true Gaussian curve). The new minimum and 
maximum are -0.70 and 17.6. The negative minimum would be set to zero and 

25 any values greater than the maximum are then deemed 1 00%. 

Another consideration is that most of the scores obtained by the operator 
wiH be around the average because we are modeling a Gaussian-Ska distribution 
using a linear model. That Is, as most of the data is centred on the mean, the 
majority of the scores will be around the mean. There is also the consideration 
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that the scopes are represented as a percentage, which no longer has a physical 
meaning. Instead, the operator will receive a score out of 10. 

The solution for the threshold problem is to calculate the thresholds in the 
office. The mean sets the lower threshold so that If the operator obtains a score 
below this then the operator Is below average. For the upper threshold, the 
threshold tor the top 10% of operators can be found. The data used to calculate 
these thresholds is all the data tor each KPI without segmentation. The threshold 
is then the average scene of fhs thresholds over the KPis. This means that we 
have a set threshold for all KPis and one that does not vary from cycle to cycle. 

The score for the KPI using the Linear Ranking Model is the ratio between 
the value and the difference of the minimum and maximum. This value Is then 
multiplied by 10 to produce the KPI score. The following equation shows the 
calculations required: 



15 



score - 10 x - 



vghte — mifitmiTrn 

maximum — minimum 



TABLE 1 below shows the advantages and disadvantages of the LRM and 
LBO methods for generating the distributions. 

TABLE 1 



Issue 


Gaussian Model 


Jnear Ranking Model 


Normal 

Gaussian 

curve 


Models this wen. 

■ i 


Wffl have a small problem in that most 
of the values concentrate around, the 
mean so it is less likely for an operator 
to achieve above 80% and less than 
20%. This can be addressed by 
owering the thresholds. Conceivably, 
these thresholds could be eel 
dynamically In the office. 


Skewed Data 
(After using 
KPis for a 
while) j 


May have a problem if a 
ot of the operators show 
an Increase in 
performance. The worst of 
he best win actually be 
aenalteed by onfy 
receiving an average 
score. 


Win handle this wed. 
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Low amount of 
data 


Wil only model the data bame problem as the Gaussian Model 
that it fe given. put can be fixed by applying manual 

Rmits. 


Spurious data 


Handles this automatically. 


Filtering will need to be applied to 
remove the outlying data. Taking the 
mean and removing any data more 
than 3 standard deviations from the 
mean wUl help this. 


tfaths 


Requires a clustering 
algorithm to model the 
data. 


Simple minimum and maximum after 
applying a simple Gaussian curve to 
filtered data. Upper and lower 
constraints can also be applied. 


Other 


Once implemented, the 
way the data is 
represented cannot be 
changed easily. 


The way the limits are calculated can 
>e changed with no changes to the 
on-board system. 



The parameters represented by KPta and their dependent parameters are: 

Swing Production - Load Weight / Swing Tune 
Swing Angle 
Hoist Dependent Swings 

Fill Production = Load Weight / (Fill + Spot TJmee) 
Start FBI Reach 
Start Fill Height 

Return Time 
Swing Angle 

4. Production Performance 

15 This is a weighted sum of the 3 KPIs above. 

9. Machine Reliability 

Hence, there are 5 KPIs and 4 different dependent parameters. The Hoiat 
20 Dependent Swings parameter does net require segmentation at all, as it is a 
Boolean. That leaves only 3 dependent parameters for which segmentation 
needs to be described. 

However, it will be appreciated that the present invention Is not limited to 
thepartlcular KPIs specified above, the number of KPIs, nor the different 
25 dependent parameters, tt is envisaged that other parameters and KPIs and 
combinations thereof may be utilized in future, depending particularly on, for 
example, the particular application. 



1. 
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In accordance with the present invention, a segmentation resolution « set 
for each dependent parameter in the database structure, except for the Hoist 
Dependent Swings parameter as previously explained. The segmentation 
resolution specifies the relevant variable(s), such as distance, angle, and the like, 
for a single segment For example, if the segmentation resolution for Swing Angle 
were 15 degrees, then data would be extracted for each 15-degree segment, as 
indicated In FIG 2. Only four segments are shown in FIG 2. A weighted sum of 
the first 3 KPIs may then be calculated to obtain an overall production 
performance rating. 

Segmentation is performed from a single known point (such as the origin 
in the case of the Start Fill Reach and Height). The data is then segmented from 
this point based on the segmentation resolution as explained above. Segments 
continue until the maximum or minimum limit is reached. 

For example, FIG 4 shows fil time data for different RU Reaches and 
Heights. In the order of darkest to Gghtest shading of the data points, the points 
represent fill times, t, of t £ 10s; 10 < t S 20s; 20 < t S 30s; and t * 30s. The 
segments would be divided suoh that they start at 0 cm and extend out to the 
10,000 cm extremity for Fill Reach. For Fill Height the segments would extend up 
to the 1,000 cm extremity and down as far as the -3,500 cm extremity. 

The reason to perform the segmentation in this way is so that the 
distributions represent a fixed set of conditions even after a period of time. This 
way, data that was logged, for example, a month ago can be fairly compared with 
current distributions. 

Another setting for the KPIs related to the segmentation is the calculation 
of a probability from the distribution. If a better performance is achieved by a 
lower KPI value, the right side of the distribution needs to be calculated to obtain 
the KPI, as shown in FIG 5. The Return Time KPI is an example of such a KPI. 
The tell side of the distribution is calculated when a KPI value is required to be 
higher to achieve better performance. The Swing Production and Fill Production 
KPIs are examples of such a KPI. 

FIG 6 shows the structure of an Integrated Mining Systems (IMS) system 
2. A Series 3 Computer Module 4 and associated Display Module 6 are located In 
each machine being monitored on site. An IMS server 8 may also be located on 
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site, for example in the efts office, or ft may be located at some other remote 
location providing communication within the Telemetry constraints Is possible. 
The IMS server 6 comprises storage means in the form of a database 10, 
calculation means in the form of KPI distribution calculation module 12, 
S communication means in the form of telemetry module 14 and application module 
16 for the generation and editing of KPI reports. 

The Database 10 also needs to store the KPI Distributions that are 
generated from the cycle data. A number of distributions are stored in the 
Database 10. The first set of Distributions model the data for that machine for ail 
10 operators. A set of Distributions wiB then exist for each operator. The feedback 
onboard can then be compared to all operators for that machine or to the 
currently logged on operator. 

An overview of the Database Structure Is described below. 

15 TABLE 2 - KPI Configuration Information 

KPI Parameter ID 

Text description of KPI 

Maximum number of Mixtures in a segment 

Left /Right distribution 

Length of moving average fitter 

The KPI Configuration information describes the global settings used in 
the system as shown in TABLE 2. The KPI Parameter ID identifies the parameter 
used In the calculation of the dtetrfcutlons and the comparisons. The text 

20 description is used to display the KPI name on the Reports/Forms. The maximum 
number of mixtures '» set here when using the LBQ method. The maximum ts 
likely to be 4, but this will probably vary depending on the KPI. The number of 
mixtures that are actually used can be smaller than this number. The Left or Right 
distribution value determines how to calculate the KPI onboard the machine. As 

25 discussed above with reference to FIG 5, if it is a left distribution, then It means 
that a higher KPI variable is required to obtain better performance, e.g. Return 
Time. A right distribution means that a lower KPI Is required to obtain better 
performance, e.g. Swing Production. A moving average filter can toe optionally 
applied to the KPI result 
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TABLE 3 Segment Information 

Contanjg 

The ID of this segment 

KP1 Parameter ID 

ID of the machine 

ID of the dig mode 

ID of the bucket 

ID of the operator 

The Segment Information contains all the combinations of machines, dig 
modes, buckets, and operators in the mine for each KPI and associated 
segments as shown in TABLE 3. The KPI Distribution Calculation routine inserts 
all the entries into this table after It has determined the segmentation of the data. 
The segment ID Identifies the segment for the current KPI. machine, dig mode, 
and the like. 

TABLE 4 Segmentation Offset Information 

Contents 

ID of the machine 

ID from Parameter Link Information 

Offset of the segment (cm, degrees, etc) 

The Segmentation Offset Information contains the offset values for 
dependent parameters associated with a KPI as shown In TABLE 4. These need 
to be configured for each machine for which KPI distribution calculations wW be 
performed. 

TABLE 5 Dependency Information 
Contents 

The IP of this segment 

The ID of the dependent parameter 

"Lower Bmit of dependent parameter 

"Higher limit of dependent parameter 

The Dependency Information contains the high and low limits for each 
dependent parameter wtthln each segment and is calculated by the KPI 
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Distribution Calculation routine. 



10 



TABLE 6 Distribution Information for the LBC3 method 



Contents 



The ID of this segment 



Mixture weight of the distribution 



Mean of the distribution 



Standard Deviation of the distribution 



The Distribution Information contains the distribution models for each of 
the segments. The information stored here depends on the distribution 
calculation method that Is employed. 

For the LBQ method, TABLE 6 shows the information that Is used. For 
each segment the mixture weight, mean and standard deviation are stored for 
each mixture within the segment. 

TABLE 7 Distribution Information for the LRM method. 



15 



Contente 



The ID of this segment 



Maximum distribution value. 



Minimum distribution value 



For the LRM method, TABUErT shows the Information that is used. For 
each segment the maximum and minimum distribution values are stored, 

TABLE 8 Parameter Link Information 



Contente 



KPI ParwneterlD 



The ID of a parameter 



Specifies whether or not the parameter is 
dependent 



The Parameter Link Information shown in TABLE 8 is used to allow 
20 parameters to be associated with a KPI- Values for associated parameters that 
are not dependent will be added to values for the KPI. Other parameters are 
dependent parameters. 
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TABLE fl Parameter Information 

Contents 
Jfhg ID of a parameter 

Text description of the parameter 

The Parameter Information shown In TABLE 0 Is used to Identify the KPI 
Parameter. ID with which the parameter is associated. This is used to identify 
5 which KPI parameter and dependent parameters are used in the modeling. 

The KPI Distribution Calculation routine Is an NT service that Is scheduled 
to run on a periodic basis. 

The program collects the data, segments it and calculates the distributions 
for each segment and stores the results in the Database 10. While this program 
10 is running, the system (mainly Telemetry module 14) knows not to acquire any of 
the data from any of the KPI tables. This is because this program may take an 
order of hours to calculate all the data. It may be necessary to set the priority of 
this task to tow In the system in case the processing time is significant 

The requirements for Telemetry are simple and would generally be familiar 
15 to a person skilled In the art. The onboard computer module 4 shown in FIO 6 
needs to request the KPI parameters that are currently in the database, but only If 
they have been changed. The onboard module 4 will request the data, for 
example, every 8 hours. If the KPI Distribution Calculation routine Is running then 
Telemetry needs to instruct the onboard module 4 to defer the request until later. 
20 It does this by setting a KPI tlmestamp in the reply packet to zero. 

The timeetemp when the data was test changed is recorded in a table in 
the database. The onboard module 4 will send an initial KPI request packet as 
described later herein. Telemetry replies with the basic KPI configuration data 
and the timestamp of when the service last ran. If the service Is running the 
25 timestamp is set to zero. The timestamp Is also sent with every packet during the 
download eo that If the service starts while downloading, the onboard module 4 
can detect that the timestamp has gone to zero and ft can abort the download. 
The Telemetry Packet Structures will now be described. 
The onboard module 4 sends a KPI Configuration Request paoket to 
30 Telemetry module 14 to request the KPI configuration. Telemetry module 14 
replies with a KPI Configuration packet, for which the contents are shown In 
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Table 10. It places the timestamp in which the KPI Distribution Calculation 
Routine last ran Into this packet The onboard module then compares this 
timestamp with the ons It has to see If It needs to start downloading the KPI 
segments. 

5 

TABLE 10 -KPI Configuration Packet 

__, Contents 

The timestamp of when the data was last updated. 

Number erf KPjs jn the database , 

The index of the KPI that we are replying to. 

KPI Parameter ID r 

Number of tapa in the Moving avenge fitter to apply to KPI 

output \ _ 

The good to excellent threshold score (%) 

The poor to good threshold score (%) 

A KPI Segment Request packet, as shown below in Table 1 1 , requests the 
data (distributions and the like) from Telemetry module 14. The reason for 
10 including the Dig Mode ID, bucket ID and the operator ID in the packet Is to 
enable priorfttzatton of the download of the KPI distributions if required. 

The first packet contains a segmentjndex of 1 to request the first segment 
and subsequent packets contain the next segment that the system wants. The 
requests stop when all the segments for that machine have been downloaded. 

15 

TABLE 11 - KPI Segment Request packet 
__ _ _____ _ — 

"KPI Parameter ID ~! 

Index to the segment for this KPI. 

The current dig mode entered on the machine. 

The current bucket on the machine. 

The currently togged on operator. 

A KPI Segment packet shown in Table 12 below Is the reply to the KPI 
segment request packet. If there is no distribution for the segment, then the 
20 Distribution information contains nothing. 



TABLE 12 - KPI Segment packet 
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Gontsnte 

The tlmestamp of when the data was last updated, 

The Total number of segments for this KPI Gncluding 
ALL dig modes and ALL buckets and ALL operators). 

KPI Parameter ID 

Dig mode ID of this distribution _ 

Bucket ID for this distribution . _ 

Operator ID for this distribution 

The Segment ID . _ 

Distribution Information 

The Production contribution of this segment 

Number of dependent parameters In this segment 

First dependent parameter ID , 

Lower limit of the dependent parameter 

Higher limit of the dependent parameter 

The Series 3 Computer Module 4 shown in FIG 6 needs to download the 
KPI configuration and distribution information from the server 8, which is stored 
onboard in Flash memory. Once this information is downloaded, performance 
indicator calculation module 18 of onboard computer module 4 is responsible tor 
calculating the KPI score* after every cycle as previously described herein. If the 
LBG algorithm method described above ts being used, a Gaussian lookup table 
may be used to calculate the Gaussian curve Instead of using the Gaussian 
distribution equation specified above. 

In order for the Series 3 Computer Module 4 to calculate the operator's 
score, ft firstly selects the distribution by determining the segment that the current 
cycle matches for the particular KPL Once the distribution has been found, then 
the KPI score can be calculated. If there exists no distribution to calculate a KPI, 
then the KPI score will be 100% (or tb If the LRM is being used). 

The scores for all the KPIs are calculated for both the mine and current 
operator comparison. Therefore, there are 2 scores that need to be calculated for 
every KPL 

The KPI can be displayed on display module 8 as a real-time parameter In 
the parameter list on a STATS screen. It may also be displayed as a trend so that 
the operator can see any performance improvements or deteriorations. The trend 
may be configured by the operator to show the graph for the tost hour or the 
current shift or other suitable period. This Is performed using the KPI trend 
configuration that is displayed once the operator selects one of the trend graphs 
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from a menu displayed on the STATS ween, 

A third option is to display a KPl Indicator that is again selected in the 
trend configuration. Throe different designs for the indicator are shown In FIGS 7- 
9 The KPl Indicator could appear white against a black background to enhance 
vtetolllty. FIG 7 shows the current real-time performance. The arrows above each 
KPl indicate whether or not the soore has improved from the last cycle. The 
extent to which the KPl has improved or deteriorated may also be shown. FIG 8 
shows an alternative method of displaying the real-time KPl scores for each of 
the KPl variables including an overall performance rating, which may be the 
average of the KPl variables, FIG 0 shows an alternative way of displaying the 
scores for the previous cycle so that the operator can judge any improvements or 
deteriorations from cycle to cycle. Thb version could include more than just the 
last cycle. 

The IMS Application module 16 preferably supports editing of at least 
some of the KPl Parameters. The following parameters need to be available to an 
administrator for editing: KPl text description; the setting of the good and 
average thresholds for the KPl Indicator; frequency of running the KPl Distribution 
Calculation routine (KPl Statistical Generator); number of days of previous data 
to be used to create the models; display of the last time the KPl data was 
updated and the lice. 

Reports, such as en Operator Performance Trend Report and an Operator 
Ranking Report, as shown in FIG 10 and FIG 11 respectively, may also be 
generated from the Report Manager In the IMS Application. 

The Operator Performance Trend report shows the graphical trend of an 
operator for each of the selected KPl variables. The options that should be made 
to the person generating this report should Include: Sort by machine, Sort by dig 
mode. Sort by bucket Set Time period, Number of operators to show (top, 
specified number or all) and The KPIs to shew. 

The Operator Performance Trend report needs to calculate the KPl values 
over the selected time period based on the distributions contained In the 
Database at the time. Therefore, the KPl scores need to be calculated again. The 
reason for this Is that the scores that were shown to the operator onboard are no 
longer valid because the distributions woufd have changed during that time and 
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therefore cannot be compared to each other. Because the Report Manager has 
to do these calculations, the report may take a long time. Therefore, the time 
period over which the trends are calculated wi! have to be limited. 

The Operator Ranking report displays the ranking of operators for each of 
5 the KPIs. That is, for a particular KPI or all KPIs, it displays the ranking of all the 
operators. The time period needs to be selected and, as for the previous report, 
this time period win have to be limited as the report may take a long time to run. 
This report needs to calculate what the previous report calculated, but needs to 
average the output scores. 
10 The options that should be made to the person generating this report 

should include: Sort by machine, Sort by dig mode. Set Time period, Number of 
operators to show (top, specified number or all), The KPIs to show. 

An Average Production KPI may be provided that may be calculated 
remotely and downloaded to the Series a computer module 6 In the machine. 
15 This may be displayed on the performance graphs to show the operator their 
current performance relative to their average. This value can be downloaded 
along with the operator ID fete. 

Current practice used by all mines estimating operator performance on the 
basis of Productivity appears to be wrong. Under different conditions and 
20 production plans some of the operators could be disadvantaged against others. 
For example, if an operator works in the same conditions, but with different swing 
angles from another operator, productivity shown for the greater swing angle will 
be less than for smaller swing angle, even though the first operator may in reality 
be more efficient 

25 Taking into account that the number of effecting factors could include a 

number of other parameters, the applicant has identified that in order to be able 
to compare productivity ranks of the same operator under different conditions, 
some integrated value that could be used for ranking purposes should be used. 
In order to be able to calculate average rank for operators working under 

30 different conditions, integrating performance ranks achieved under different 
conditions by different operators should be considered on the one hand and mine 
Interests and production performance should be considered on another hand. 
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The suggested method of the present Invention In this regard will Include 
these 2 parameters as variables and will allow calculation of average operator 
rank, which could be used as a universal rank among the mine for different 
machines, conditions and production plans. 

The formula for calculation of average operator rank is presented below: 
Av Op Rank «Wi*Ri + W 2 *Ra+.".+ WfRi 

where: 

Wj - Weight coefficient tor Parameter Subset i, whtch Is calculated on the basis of 
statistical information for the mine Indicating the weight of i Parameter subset for 
the mine applicable to operator I; and 

Ri - Rank of the operator i achieved for this Parameter Subset 1. 

For example, let It be assumed that during a reporting period a mine used 
only four different subsets of parameters. The weight of each subset could 
respectively be the following: 25%, 20%, 40% and15%. If operator #1 worked 
only under subset #1 and # 2 and achieved 90% for subset # 1 and 94% for 
subset #2, using the above formula the average rank for the operator may be 
calculated: 

Av Op Rank -H*90%+^*94%-91.8% 
45 45 

For Operator # 2, subset #3 - 02% and subset #4 = 90%. Hence: 
Av Op Rank -gx92%+i|x90% - 91.45% 

These Productivity ranks do not Include Production figures and only rank 
operators for different subsets of parameters. In reality, if, for example, operator 
#1 was doing cycles with swings of say 10 and 20 degrees and operator #2 
swings of say 170 and 180 degrees, then the real production for operator #1 
could be twice as much as for operator # 2. but In fact the rank of operator # 1 Is 
higher and accordingly he Is better 

H is also conceivable that the average performance of an operator over the 
last week or month could be shown. The average performance could be 
calculated remotely and the onboard rrxxdula would download it to the machine 
for every operator. It would be treated just as a list download where one radio 
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packet represents one graph. Only the minimum and maximum values need to be 
sent and then each of the data points can be percentage scaled. 

Accurately determining one or more of the KP!s in accordance with the 
present invention addresses the difficulties of accurately measuring relevant 
5 parameters and producing fair comparisons. The present invention can be uBed 
to improve awareness of how wen the operators are performing and provide an 
incentive to improve performance. It also provides an Indication to management 
about who Is performing wed and which operators are not performing up to 
standard. 

10 Throughout the specification the aim has been to describe the invention 

without limiting the Invention to any one embodiment or specific collection of 
features. Persons skilled In the relevant art may realize variations from the 
specific embodiments that will nonetheless fal within the scope of the Invention. 

15 
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CLAIMS: 

1. A method for monitoring performance of at least one machine operator, 
said method Including the steps oft 

measuring at least one machine parameter during operation of the 
machine by the operator 

generating at least one performance indicator distribution from 
measurements of the at least one machine parameter and, 
calculating at least one performance Indicator from the at least one 
performance indicator distribution. 

2. The method of claim 1 . further including the step of providing feedback to 
the operator by displaying the at least one performance indicator in 
substantially reaMSme to the operator. 

3. The method of claim 1 . further including the step of providing feedback to 
the operator by displaying the at least one performance Indicator to the 
operator once the machine has completed an operation cycle. 

4. The method of daim 1, wherein the at least one machine parameter is a 
dependent machine parameter. 

6. The method of claim 1, wherein the at least one machine parameter Is 
the sote parameter represented by a particular performance Indicator. 

a The method of claim 4, further including the step of segmenting at least 
one of the dependent machine parameters into segments, the range of 
each segment constituting a segmentation resolution. 

7. The method of claim 6, wherein the step of segmenting at teas* one of 
the dependent machine parameters includes specifying a magnitude of 
the range for each segment of each dependent machine parameter 
requiring segmentation. 



* 
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8. The method of claim 4, wherein at least one dependent machine 
parameter does not require segmentation. 

9, The method of claim 1, wherein the step of generating the at least one 
performance indicator distribution include* using a mixture of one or 
more distributions to model the perfbimanoe indicator distribution. 

10. The method of claim 0, wherein the number of mixture* is set 
dynamically. 

1 1, The method of claim 1, wherein the at least one performance indicator 
distribution is generated using an algorithm. 

12- The method of claim 11. wherein the algorithm is a LJnde*uzo-Gray 
(LBG) algorithm. 

13. The method of claim 1. wherein the at least one performance Indicator 
dlstrftiution Is generated using a linear ranking model (LRM). 

14. The method of claim 1, wherein two or more performance indicators am 
combined to yield an overai performance rating of the machine operator. 

15. The method of claim 14, wherein one or more of the performance 
indicators are positively or negatively weighted with respect to the other 
performance Indicators). 

18. A system for monitoring performance of at least one machine operator, 
said system comprising: 

at least one measuring device for measuring at least one machine 
parameter during operation of the machine by the operator, 
a server for generating at least one performance Indicator distribution 
from measurements of the at least one machine parameter and, 
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a performance indicator calculation module tor calculating at least one 
performance indicator from the at least one performance Indicator 
distribution. 

17. The system of claim 16, wherein the server Is remote from the 
machine. 

1 B.The system of claim 1 6, wherein the server comprises: 
storage means; 
commirication means; and 

a perfbrmanoe indicator distribution calculation module. 

19. The system of claim 16, wherein the performance indicator calculation 
module is onboard the machine. 

20. "me eystem of claim 16, wherein the performance indicator calculation 
module is coupled to communication means for transmitting and 
receiving data to and from the server. 

21- The system of daim 16, further comprising at least one display device. 

22. The system of claim 21, wherein the at least one display device 
displays the at least one performance Indicator in substantially real-time 
to the operator* 

23. The system of claim 21. whBiein the at toast one display device 
displays the at least one performance indicator to the operator once the 
machine has completed an operation cycle. 

24. The system of claim 21, wherein the at least one display device is 
onboard the machine. 



25. The system of claim 21, wherein the at toast one display device is 
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